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EXECUTIVE  SUMMARY 


The  purpose  of  this  study  was  to  conduct  a  preliminary  evaluation 
concerning  the  impact  of  a  learning  strategies  program  on  soldier  performance. 
This  learning  strategy  utilized  Feuerstein's  Instrumental  Enrichment  (IE) 
materials  incorporated  into  the  Basic  Skills  Education  Program  (BSEP)  being 
conducted  at  Fort  Knox.  The  objective  of  the  BSEP/IE  program  was  to  improve 
the  cognitive  skills  of  soldiers  and  in  turn  their  job  performance. 

Soldiers  participating  in  Fort  Knox  BSEP  classes  during  the  Fall  of 
1981  were  the  primary  subjects  for  the  study.  On  the  basis  of  a  comparison 
of  Pre-  and  Post-Test  scores  for  soldiers  that  participated  in  conventional 
BSEP  classes  (in  the  Fall  of  1980)  and  those  participating  in  BSEP  incorpor¬ 
ating  IE,  it  appears  that  IE  resulted  in  a  small  but  reliable  improvement 
in  test  scores.  With  regard  to  job  performance,  supervisor  ratings  suggest 
that  improvements  occurred  in  some  iMntal  activities,  spatial  orientation, 
planning,  and  working  independently.  It  should  be  noted,  however,  that  no 
firm  conclusion  about  the  effects  of  IE  can  be  made  due  to  the  lack  of  an 
experimental  and  control  group  formed  through  random  assignment.  Since  it 
appears  that  IE  did  have  some  impact,  it  is  recommended  that  a  more  rigorously 
controlled  evaluation  study  be  performed.  A  conceptual  plan  for  conducting 
such  a  study  is  included  in  this  report. 
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INTRODUCTION 


One  of  the  major  objectives  of  Army  training  is  to  strengthen 
the  basic  skills  of  soldiers  who  are  deficient  in  this  respect.  In  keeping 
with  this  objective,  the  Army  Training  Developments  Institute  funded  an 
exploratory  effort  to  evaluate  a  learning  strategies  concept  developed  by 
R.  Feuerstein.  Feuerstein  identified  22  cognitive  functions  that  he  believed 
are  critical  to  the  learning  process.  With  these  cognitive  functions  as  the 
basis,  Feuerstein  developed  the  Instrumental  Enrichment  (.IE)  Program.  The 
program  consists  of  14  instruments  which  are  in  the  form  of  paper  and  pencil 
exercises.  Each  instrument  consists  of  exercises  which  are  directed  toward 
overcoming  a  specific  cognitive  deficiency  (Hobbs,  1980).  Another  salient 
feature  of  IE  is  that  it  is  designed  to  provide  the  basis  for  learning.  The 
cognitive  skills  that  the  IE  program  is  designed  to  teach  include  organizing, 
planning,  synthesizing,  problem  definition,  perceiving  relationships, 
analytical  thinking,  and  systematic  searching.  Experience  with  the  program 
has  indicated  that  it  has  had  a  positive  impact  on  culturally  deprived  students 
who  have  had  limited  learning  opportunities.  For  example,  participation  in  the 
program  has  improved  academic  achievement  and  increased  scores  on  standardized 
tests  (Curriculum  Development  Associates). 

To  assess  the  applicability  of  the  IE  program  to  soldier  performance, 
the  Army  introduced  components  of  the  IE  program  into  its  Basic  Skills  Education 


Program  (BSEP)  at  Fort  Knox,  Kentucky.  On  the  basis  of  preliminary  needs 
assessment  vwrk  (Link,  1981),  nine  of  Feuerstein's  14  IE  instruments  or  work¬ 
books  were  selected  for  incorporation  into  the  BSEP  program.  In  general, 
approximately  two  hours  per  day  were  spent  each  day  throughout  both  the  six 
and  12  week  BSEP  sessions.  According  to  the  teaching  contractor'*,  all  but  a 
few  students  completed  four  of  the  workbooks  (i.e..  Organization  of  Dots, 
Orientation  in  Space,  Comparisons,  and  Analytical  Perceptions).  Some  of  the 
faster  students  completed  additional  workbooks  (i.e..  Instructions,  Numerical 
Progressions,  Illustrations  and  Categorization).  To  assess  the  value  of 
incorporating  the  IE  program  into  BSEP,  the  Amty  funded  studies  to  evaluate  its 
impact  on  soldier  cognitive  skills  and  on  soldier  job  performance**.  The 
present  study  was  designed  to  evaluate  the  latter. 


*  Elizabethtown  Conmunity  College. 

**  The  cognitive  skills  study  is  being  carried  out  by  the  American  Institute 
for  Research,  Palo  Alto,  California  under  contract  with  the  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 


STUDY  OBJECTIVES 


The  major  purpose  of  the  present  study  was  to  conduct  a  preliminary 
evaluation  of  the  impact  of  IE  materials  on  soldier  performance.  More 
specifically,  the  objectives  of  the  study  were: 

•  To  determine  whether  IE  has  an  influence  on  soldier 
BSEP  performance.  That  is,  if  cognitive  skills  are 
improved  during  the  course  of  BSEP,  then  there  should 
be  a  positive  influence  on  BSEP  scores  and  on  Post- 
BSEP  TABE*  scores. 

•  To  determine  the  combined  effect  of  IE  and  BSEP  on 
soldier  job  performance.  It  is  hypothesized  that  IE 
will  improve  cognitive  skills,  which  in  turn  will 
improve  soldier  performance  on  the  job. 

The  methodology  used  in  accomplishing  these  objectives  is  presented  in 
the  following  section  of  this  report. 


*  Test  of  Adult  Basic  Education. 
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METHODOLOGY 


The  project  consisted  of  two  basic  studies:  (1)  a  comparison  of  the 
performance  of  soldiers  who  had  BSEP  that  Incorporated  IE  to  soldiers  who  had 
BSEP  without  IE,  and  (2)  a  questionnaire  survey  of  the  supervisors  of  soldiers 
who  had  BSEP  with  IE  concerning  the  soldiers'  job  performance.  The  two  studies 
are  discussed  below. 


IE  Effects  on  BSEP  Performance 


Study  Design 

The  basic  design  of  this  study  was  the  comparison  of  soldiers  who 
took  BSEP  with  IE  to  soldiers  who  took  BSEP  without  IE.  Two  groups  of  soldiers 
who  had  BSEP  with  IE  were  Identified  at  Fort  Knox.  One  group  participated 
from  8  September,  1981  to  2  November,  1981  (termed  the  first  cohort)  and  the 
second  group  from  9  November,  1981  to  18  December,  1981  (termed  the  second 
cohort).  A  group  of  soldiers  (termed  the  control  grouo)  who  had  BSEP  without 
IE,  took  the  course  In  the  Fall  of  1980,  or  about  a  year  earlier  than  the  first 
and  second  cohorts. 

A  second  BSEP  control  group  was  also  Identified  and  used.  Based  on 
discussions  with  Army  personnel,  the  BSEP  program  at  Fort  Sill,  Oklahoma  was 
deemed  sufficiently  comparable  to  BSEP  at  Fort  Knox.  Consequently,  a  group  of 
soldiers  who  had  taken  BSEP  without  IE  during  the  Fall  of  1981  were  used  as  the 
second  control  group. 

Performance  In  BSEP  Is  measured  by  the  TABE  test.  Both  Pre-  and 
Post-  measures  are  taken  on  each  soldier  with  subscales  on: 

•  Reading  -  Vocabulary 

•  Reading  -  Comprehension 

•  Mathematics  -  Computation 

•  Mathematics  -  Concepts  and  Problems 

•  Language  -  Mechanics  and  Expression 

•  Spelling 

These  subscales  scores  are  then  combined  to  yield  totals  for  Reading  and  Mathe¬ 
matics,  as  well  as  Total  Battery  score.  All  scores  are  expressed  as  Grade 
Equivalent  scores. 


To  measure  the  influence  of  IE  on  TABE  scores,  Post-BSEP  TABE  scores 
were  compared  for  the  combined  first  and  second  cohorts  and  each  of  the  two 
control  groups  controlling  for  Pre-BSEP  TABE  scores.  The  hypothesis  was  that 
if  IE,  in  fact,  improves  cognitive  skills  the  two  BSEP/IE  cohorts  will  have 
higher  scores  after  BSEP  than  either  of  the  two  control  groups. 

This  study  design  does  have  limitations.  From  the  outset,  it  was 
recognized  that  the  control  groups  were  not  optimal.  Ideally,  the  control 
group  should  have  been  a  group  of  soldiers  given  BSEP  without  IE  selected  at 
the  same  time  as  the  first  and  second  cohorts  and  taught  by  the  same  teachers. 
However,  this  was  not  possible.  Consequently,  the  most  suitable  approach  was 
to  select  a  group  of  soldiers  for  the  control  group  having  BSEP  without  IE 
taught  by  the  same  teachers  as  the  BSEP/IE  group.  The  basic  difference  (other 
than  receiving  IE)  between  the  control  group  used  and  the  two  cohorts  is  that 
the  control  group  received  more  BSEP  material.  The  control  group  received  four 
hours  per  day  of  basic  skills  material  and  MOS-related  material.  The  two 
cohorts  received  two  hours  of  basic  skills  material  and  MOS-related  material 
and  two  hours  of  IE.  Thus,  it  is  conceivable  that  the  control  group  may  achieve 
higher  gains  on  the  TABE  than  the  two  cohorts  simply  because  they  had  more  BSEP 
material  each  day.  But,  if  the  two  cohorts  achieve  higher  gain  scores  than  the 
controls,  then  it  can  be  concluded  that  IE  did  have  a  positive  influence.  If 
therp  is  no  difference,  then  the  addition  of  IE  may  have  compensated  for  the 
decreased  BSEP  material. 

Data  Collection 


The  data  collected  for  the  two  cohorts  and  the  Fort  Knox  control 
groups  were  Pre-  and  Post-BSEP  TABE  scores  and  Pre-BSEP  GT  scores  for  each 
soldier.  The  scores  were  received  for  a  total  of  69  soldiers  for  the  first 
cohort,  100  soldiers  for  the  second  cohort,  and  58  soldiers  for  the  control 
group.  (Additional  data  collected,  but  not  used  were  the  Name,  Rank,  Social 
Security  Number  and  Unit  Classification  for  each  soldier.)  Data  received  from 
Fort  Sill  contained  only  the  Pre-  and  Post-BSEP  TABE  scores  for  69  soldiers. 
Fort  Sill  did  not  report  the  Total  Battery  scores.  The  data  were  obtained  from 
Fort  Knox  and  Fort  Sill  BSEP  administrators. 
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B$EP/IE  Effects  on  Job  Performance 


Study  Design 

It  was  hypothesized  that  if  IE  improves  soldier  cognitive  skills, 
then  certain  aspects  of  the  soldier's  performance  should  also  be  improved. 
However,  there  is  a  confounding  factor.  BSEP  should  also  improve  cognitive 
skills,  and  since  soldiers  receive  both  BSEP  and  IE  together  there  is  no  way 
to  separate  the  effects  of  the  two  on  job  performance.  Consequently,  the 
study  examined  job  performance  as  a  result  of  IE  and  BSEP  in  combination. 

Soldier  job  performance  was  evaluated  through  a  questionnaire  survey 
of  the  two  cohorts'  supervisors*.  The  basic  design  was  to  have  a  soldier's 
job  performance  evaluated  prior  to  BSEP/IE  and  six  weeks  or  so  after  BSEP/IE. 
The  six  week  length  was  selected  as  a  compromise  between  the  desire  for  the 
supervisors  to  have  sufficient  time  to  observe  and  interact  with  the  soldier 
after  BSEP/IE,  and  the  short  schedule  for  project  completion.  Also,  because 
of  the  timing  of  the  study,  there  was  no  way  to  distribute  a  questionnaire 
prior  to  a  soldier's  involvement  in  BSEP/IE.  Consequently,  both  Pre-BSEP  and 
Post-BSEP  questionnaires  had  to  be  distributed  after  BSEP/IE  completion. 

The  questionnaires  were  developed  by  first  identifying  job  perform¬ 
ance  parameters  which  were  representative  of  those  cognitive  skills  that  IE 
was  expected  to  improve.  Previous  reports  were  used  to  identify  those  cogni¬ 
tive  skills  (Hobbs,  1980;  Link,  1981).  These  were  then  translated  into  cogni¬ 
tive/performance  parameters  that  were  judged  to  be  relevant  to  a  soldier's 
job  performance.  The  parameters  identified  were: 

•  Problem  Solving 

•  Accuracy  and  Precision 

•  Verbal  Naming 

•  Learning  New  Tasks 

t  Planning 


*  Job  performance  as  a  result  of  BSEP  without  IE  was  not  examined  because 
of  the  probable  lack  of  sufficient  number  of  soldiers  whose  current  or 
previous  supervisor  could  make  such  a  judgment. 
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•  Spatial  Orientation  and  Direction 

•  Following  Instructions 

•  Relationships 

•  Checking  Own  Work 

•  Volunteering 

•  Attention  and  Concentration 

•  Independence 

•  Illustrations 

For  each  of  the  parameters,  a  set  of  two  to  four  behavioral ly  oriented  job 
statements  were  developed.  For  example,  the  parameter  "Spatial  Orientation  and 
Direction"  had  the  following  statements: 

•  Gets  to  parts  of  the  Post  without  asking  for 
directions 

•  Identifies  North,  South,  East  and  West 
correctly 

•  Understands  how  parts  of  equipment  used  on 
the  job  fit  or  connect  together 

•  Utilizes  maps  and/or  diagrams  correctly 

Four  statements  that  did  not  deal  with  job  performance  were  also 
included  in  the  questionnaire  to  check  on  the  tendency  of  supervisors  to  rate 
all  items  higher  (or  lower)  on  the  second  administration.  A  total  of  39 
statements,  35  of  which  related  to  job  performance,  were  prepared.  The 
behavioral  statements  for  all  the  parameters  are  included  in  Appendix  I. 

To  rank  the  soldiers  on  each  of  the  behavioral  statements  a  seven 
point  scale  was  developed.  The  scale  ranged  from  1  to  7,  with  1  representing 
"Almost  Never",  4  representing  "Half  the  Time",  and  7  representing  "Almost 
Always".  The  other  scale  numbers  were  not  labeled.  The  supervisors  were 
asked  to  circle  the  number  best  describing  the  frequency  with  which  the 
soldier  met  the  behavioral  statement. 

To  allow  the  supervisors  an  opportunity  to  voice  an  opinion  on 
observed  differences'  in  the  soldier's  performance,  an  open-ended  question  was 
included  in  the  Post-BSEP/IE  questionnaire.  The  question  was,  "How  would  you 
describe  the  difference  in  this  soldier's  performance  between  a  month  before 
BSEP  and  the  past  few  weeks?  For  example,  what  differences  have  you  noticed 
in  mental  performance,  and  in  job  performance?"  The  question  was  placed  at 
the  end  of  the  questionnaire. 


Finally,  a  set  of  instructions  were  developed  for  the  supervisors 
to  explain  procedures  for  completing  the  questionnaires.  Another  set  of 
instructions  were  prepared  for  the  Unit  Commander's  Office  to  explain  distri¬ 
bution.  These  instructions  and  the  questionnaires  are  presented  in  Appendix  II. 

Data  Co 1 lecti on 


The  data  collection  entailed  the  distribution  of  the  Pre-BSEP  and 
Post-BSEP  questionnaires  to  the  supervisors  of  the  two  cohorts.  The  first 
distribution  occurred  mid-December,  1981,  which  was  about  six  weeks  after  the 
first  cohort  had  finished  BSEP.  The  Pre-BSEP  questionnaires  were  distributed 
to  individuals  (usually  the  Adjutant)  at  the  Unit  Commander  Offices.  They  in 
turn  distributed  the  questionnaires  to  the  supervisors.  The  Post-BSEP  ques¬ 
tionnaires  were  distributed  to  the  Commander's  Office  on  the  second  day.  All 
questionnaires  were  returned  to  the  Education  Office  and  then  to  Battel le. 

In  early  February  1982,  the  same  approach  was  used  to  collect  the  questionnaire 
data  for  the  second  cohort.  This  was  about  six  weeks  after  the  second  cohort 
had  finished  BSEP.  The  separate  distribution  of  the  Pre-  and  Post-BSEP 
questionnaires  was  intended  to  insure  that  the  supervisors  would  not  fill  out 
both  questionnaires  side-by-side.  However,  there  was  no  way  to  preclude  this 
from  occurring  and  because  of  the  short  project  time  schedule,  there  was  no 
other  cost-effective  distribution  approach  to  insure  independent  completion 
of  the  two  questionnaires. 

A  total  of  372  Pre-  and  Post-BSEP  questionnaires  were  distributed 
and  228  were  returned,  i.e.,  about  a  61*  return  rate.  There  were  several 
reasons  given  for  not  returning  a  questionnaire: 

•  The  soldier  had  left  the  post, 

•  The  soldier  had  a  new  supervisor, 

•  The  supervisor  was  on  TDY, 

•  The  supervisor  chose  not  to  fill  out  the 
questionnaire.  (Questionnaire  instructions 
indicated  that  completion  was  voluntary.) 

The  61*  return  rate,  however,  did  not  represent  the  uaeable  return.  Many 
questionnaires  were  not  useable  for  the  following  reasons: 
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•  The  soldier  had  left  the  post  and  the  question¬ 
naire  was  returned  incomplete, 

•  The  soldier's  supervisor  had  changed  and  the 
questionnaire  was  returned  incomplete, 

•  The  Post-BSEP  questionnaire  was  returned,  but 
filled  out  by  a  different  supervisor  than  the 
Pre-BSEP  supervisor, 

•  The  Pre-BSEP  questionnaire  was  returned,  but 
the  Post-BSEP  questionnaire  was  not  and  vice- 
versa. 

For  these  reasons,  about  41%  of  the  returned  questionnaires  could  not  be  used 
in  the  data  analysis.  A  total  of  66  pairs  of  questionnaires  were  used  for 
purposes  of  data  analysis. 

The  questionnaires  were  developed  so  that  the  data  did  not  require 
editing  before  data  processing.  However,  the  open-ended  question  at  the  end 
of  the  Post-BSEP  questionnaire  did  require  editing.  Two  aspects  of  the 
supervisors'  answers  were  examined.  The  first  aspect  was  an  indication  that 
the  soldier  did  or  did  not  improve  in  performance  as  a  result  of  BSEP/IE,  and, 
second,  if  there  was  improvement,  a  determination  of  the  associated  cognitive 
or  performance  parameter.  The  following  coding  scheme  was  developed  for  each 
answer: 

Indiaatione  of  Improvement 

•  Blank  or  not  enough  opportunity  to  observe 

•  No  difference 

•  Degraded  performance  or  attitude 

0  Good  comments  only  (no  indication  of  change) 

0  Improvement  from  job  experience  only,  other¬ 
wise  no  improvement  as  a  result  of  BSEP/IE 

0  Improvement  presumably  due  to  BSEP/IE 

Areoa  of  Improvement* 

0  Duty  Performance/Increased  Production 

0  Mental  Performance 

0  Communication  (writing,  speaking) 

0  Reading 

0  Attitude/Interest/Enthusiasm 


*  Up  to  three  areas  of  improvement  were  considered. 
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•  Confidence/ Assertion/Taking  Charge/Helping  Others 
a  Planning/Use  of  Time 

•  Concentration/Attention  to  Details 
a  Accuracy 

a  Responsibility 

a  Understanding  of  Job  and  Equipment 
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RESULTS 


The  analyses  of  the  results  are  presented  below  in  two  main  sec¬ 
tions;  IE  effects  on  BSEP  performance  as  measured  with  TABE,  and  BSEP/IE 
effects  on  job  performance  as  measured  by  the  job  performance  questionnaires. 

IE  Effects  on  BSEP  Performance 


Comparison  of  First  and  Second  Cohorts 

The  two  cohorts  who  participated  in  the  BSEP/IE  program  for  six 
weeks  at  Fort  Knox  in  the  Fall  of  1981  were  compared  on  Pre-BSEP  TABE  scores, 
Post-BSEP  TABE  scores  and  Pre-BSEP  GT  scores.  The  groups  were  compared  using 
Student's  t-test.  All  comparisons  were  non-significant*.  Based  on  these 
results,  the  two  cohorts  were  combined  in  subsequent  analyses. 

Comparison  of  BSEP/IE  Participants  with 
Fort  Sill  BSEP  Participants' 

The  TABE  scores  of  soldiers  who  participated  in  BSEP/IE  at  Fort 
Knox  in  the  Fall  of  1981  were  compared  with  the  scores  of  soldiers  who  parti¬ 
cipated  in  the  BSEP  program  at  Fort  Sill  in  the  same  time  period.  At  the 
onset,  it  was  recognized  that  the  two  groups  differed  in  several  respects 
(notably  rank  and  MOS).  In  addition,  the  BSEP  programs  and  selection  criteria 
also  differed.  Nevertheless,  there  was  interest  in  using  the  Fort  Sill  group 
as  a  possible  control  group. 

The  groups  were  compared  using  the  analysis  of  covariance  (ANCOVA). 
This  powerful  form  of  analysis  compares  the  Post-BSEP  TABE  means  of  the  two 
groups  after  adjusting  for  the  corresponding  Pre-BSEP  TABE  means.  The  analysis 
assumes  that  the  relationship  between  Pre-  and  Post-BSEP  TABE  scores  is  the 
same  for  both  groups.  Table  1  presents  the  results  of  the  ANCOVA 's  for  each 
subscale**. 


*  In  all  analyses,  the  five  percent  level  of  significance  was  used  as  the 
minimum  significance  level. 

'*  The  Fort  Sill  data  did  not  include  the  Total  Battery  TABE  scores. 
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Prior  to  BSEP,  the  groups  differed  significantly  on  the  three  Reading  subscales, 
and  on  the  Mathematics  Concepts  and  Problems  subscale,  with  the  Fort  Knox 
participants  showing  higher  average  Pre-BSEP  TABE  scores.  After  BSEP,  the 
groups  differed  significantly  on  the  Mathematics  Computation  subscale,  with 
the  Fort  Sill  participants  showing  higher  average  Post-BSEP  TABE  scores. 

When  the  Post-BSEP  TABE  means  are  adjusted  for  the  corresponding 
Pre-BSEP  TABE  means  (the  covariate  control),  the  Fort  Sill  participants  had 
significantly  higher  means  on  Reading  Vacabulary,  Reading  Total,  Mathematics 
Computation,  Mathematics  Concepts  and  Problems,  Mathematics  Total,  and  Language 
Mechanics  and  Expression  subscales  (see  Table  1).  However,  the  relationship 
between  Pre-  and  Post-BSEP  TABE  scores  was  not  the  same  for  both  groups.  The 
Fort  Sill  participant  scores  showed  significantly  higher  correlation  between 
Pre-  and  Post-BSEP  on  five  subscales,  as  shown  in  Table  1.  This  effect  appears 
as  an  interaction  of  groups  with  the  covariate. 

A  form  of  the  Johnston-Neyman  test  was  used  to  test  whether  the  means 
on  these  five  subscales  differed  significantly  for  an  average  Pre-BSEP  TABE 
subscale  score  after  adjustment  for  Pre-BSEP  TABE  means  and  the  interaction  of 
groups  with  the  covariate.  The  five  pairs  of  adjusted  means  were  significantly 
different  from  each  other  (see  Table  1),  with  the  Fort  Sill  participants  show¬ 
ing  significantly  higher  means.  Thus,  only  in  the  areas  of  Reading  Comprehen¬ 
sion  and  Spelling  were  the  adjusted  TABE  means  not  significantly  different. 

The  ANCOVA's  presented  in  Table  1  indicate  that  the  Fort  Sill  and 
Fort  Knox  participants  were  quite  different,  both  before  participating  in  BSEP 
and  in  the  amount  of  gain  after  BSEP.  These  differences  probably  reflect 
differences  in  BSEP  selection  procedures  between  the  two  posts,  as  well  as 
differences  in  the  BSEP  programs  themselves.  For  these  reasons,  the  Fort  Sill 
participants  do  not  appear  to  be  an  acceptable  control  group.  Certainly  the 
comparison  does  not  support  the  superiority  of  BSEP  with  IE. 

Comparison  of  BSEP/IE  Participants  with 
Earlier  Eort  Knox  BSEP  (Participants 


The  TABE  scores  of  soldiers  who  participated  in  BSEP/IE  at  Fort  Knox 
in  the  Fall  of  1981  were  compared  with  the  scores  of  soldiers  who  participated 
in  the  BSEP  program  at  Fort  Knox  in  the  Fall  of  1980.  The  groups  were  compared 
using  the  ANCOVA.  Table  2  presents  the  results  of  the  ANCOVA's  for  each  sub¬ 
scale  and  the  Total  Battery.  Prior  to  BSEP,  the  groups  differed  significantly 
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TABLE  2.  COMPARISON  OF  GRADE  EQUIVALENT  SCORES  ON  TABE  TEST  BATTERY  FOR 
FORT  KNOX,  FALL  1981  AND  FORT  KNOX,  FALL  1980  BSEP  PARTICIPANTS 
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N.S.  -  not  significant 


on  only  one  subscale.  Language  Mechanics  and  Expression.  Following  BSEP,  the 
groups  differed  significantly  on  Reading  Total  and  Language  Mechanics  and 
Expression. 

When  the  Post-BSEP  TABE  means  are  adjusted  for  the  corresponding  Pre- 
BSEP  TABE  means  (the  covariate  control),  the  BSEP/IE  participants  had  signifi¬ 
cantly  higher  means  than  the  control  group  on  Language  Mechanics  and  Expression, 
and  Total  Battery  (see  Table  2).  No  significant  interactions  of  group  with  the 
covariate  were  found,  indicating  that  the  regression  with  the  covariate  did  not 
differ  significantly  in  the  two  groups. 

The  ANCOVA's  presented  in  Table  2  indicate  a  significant  Total  Battery 
TABE  difference  between  the  groups  following  BSEP,  with  the  difference  favoring 
those  participants  who  received  IE.  Based  on  the  lack  of  appreciable  differ¬ 
ences  in  Pre-BSEP  TABE  means  and  the  absence  of  significant  interactions  of 
group  with  covariate,  it  1s  concluded  that  the  Fall,  1980  Fort  Knox  group 
provided  a  suitable  control  group.  Thus,  it  appears  that  IE  had  some  small  but 
reliable  impact  on  Total  Battery  TABE  scores.  The  impact  was  largely  in  the 
area  of  Language  Mechanics  and  Expression. 

It  should  be  noted  that  the  BSEP/IE  participants  achieved  equal  or 
higher  means  than  the  control  group,  even  though  the  amount  of  time  devoted  to 
BSEP  in  the  Fall  of  1981  was  only  about  half  of  what  was  spent  on  BSEP  in  the 
Fall  of  1980. 

Comparison  of  Fort  Knox  Groups  When 
GT  Scores  Are  Included 


The  inclusion  of  the  Pre-BSEP  GT  scores  as  a  second  covariate  in  the 
ANCOVA's  presented  in  Table  2  did  not  change  any  of  the  results  reported,  but 
did  improve  the  sensitivity  of  the  analyses.  Pre-BSEP  GT  scores  are  signifi¬ 
cantly  correlated  with  Pre-  and  Post-BSEP  TABE  scores.  Pre-BSEP  GT  scores  are 
also  significantly  correlated  with  Post-BSEP  TABE  scores,  after  adjusting  these 
latter  scores  for  (partialling  out)  the  correlation  with  the  corresponding  Pre- 
BSEP  TABE  scores,  for  Reading  Vocabulary,  Reading  Comprehension,  Mathematics 
Concepts  and  Problems,  Mathematics  Total  and  Total  Battery  scores.  Thus,  Pre- 
BSEP  GT  scores  can  provide  an  additional  pre-treatment  control  in  future  studies. 

It  has  been  hypothesized  that  the  GT  score  is  predictive  of  the  gains 
a  soldier  will  experience  in  BSEP.  There  is  some  statistical  truth  in  the 
hypothesis,  for  as  noted  above,  the  Pre-BSEP  GT  scores  are  significantly 
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correlated  with  Post-BESP  TABE  scores,  even  after  adjusting  for  Pre-BSEP  TABE 
scores.  The  relationship  is  slight,  however.  Pre-BSEP  GT  scores,  for  example, 
account  for  only  16  percent  of  the  variance  in  Post-BSEP  Total  Battery  scores. 
Pre-BSEP  GT  scores  are  not  significantly  correlated  with  TABE  gain  scores  (Post- 
BSEP  TABE  scores  minus  Pre-BSEP  TABE  scores)  on  any  subscale  or  for  the  Total 
Battery  score. 


BSEP/IE  Effects  on  Job  Performance 


The  effects  of  BSEP  with  an  IE  component  on  soldiers'  job  performance 
were  evaluated  through  a  questionnaire  survey  of  the  participants'  supervisors. 

No  control  group  was  involved*.  The  survey  consisted  of  39  behaviorally- 
oriented  job  performance  statements  that  were  rated  by  the  supervisors  on  a 
seven-point  scale.  Two  sets  of  ratings  were  obtained;  one  set  representing 
Pre-BSEP  job  performance,  and  one  set  representing  job  performance  about  six 
weeks  after  BSEP  completion. 

Description  of  the  Ratings 

The  obtained  ratings  were  highly  skewed  toward  the  favorable  end  of 
the  scale.  The  mean  Pre-BSEP  rating  was  5.62.  This  allowed  little  room  for 
improvement  on  subsequent  ratings.  This  type  of  leniency  error  is  common  in 
supervisor's  ratings. 

The  ratings  on  the  39  statements  showed  substantial  intercorrelations. 
The  Pre-BSEP  questionnaire  and  the  Post-BSEP  questionnaire  were  subjected  to 
factor  analyses  to  determine  if  there  were  significant  grouping  of  statements. 

The  results  indicated  that  the  first  factor  extracted  in  the  analysis  of  the  Pre- 
BSEP  statements  accounted  for  61  percent  of  the  total  variance  among  statements, 
whereas  the  first  factor  extracted  in  the  analysis  of  the  Post-BSEP  statements 
accounted  for  68  percent  of  the  total  variance  among  statements.  Other  factors 
each  accounted  for  less  than  six  percent  of  the  total  variance.  These  results 
preclude  any  empirical  grouping  of  statements  and  suggest  a  strong  halo  effect 
in  the  ratings. 

*  An  appropriate  control  group  would  consist  of  soldiers  who  were  eligible  for 
BSEP  but  who  had  not  participated.  It  was  Battelle's  judgment  that  an  insuf¬ 
ficient  number  of  these  soldiers  would  be  available.  Further,  sufficient  time 
was  not  available  to  identify  these  soldiers  and  collect  job  performance 
data  for  them. 
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Formation  of  Subscales  and  Total  Score 


The  39  statements  were  grouped  into  14  areas  or  subscales  (see 
Appendix  I),  including  a  subscale  of  four  irrelevant  (non -job-performance) 
statements.  Each  subscale  consisted  of  two  to  four  statements.  A  supervi¬ 
sor's  ratings  on  statements  assigned  to  a  subscale  were  averaged  to  provide  a 
Pre-BSEP  and  Post-BSEP  subscale  score.  The  ratings  on  the  35  job  performance 
statements  (i.e.,  not  including  the  ratings  on  the  four  irrelevant  items)  were 
averaged  to  provide  a  Pre-BSEP  and  a  Post-BSEP  total  score. 

Comparison  of  First  and  Second  Cohorts 

The  job  performance  ratings  of  the  two  cohorts  who  participated  in 
the  BSEP/IE  program  were  compared  using  Student's  t-test.  The  groups  were 
compared  statement  by  statement,  on  subscales,  and  on  total  score.  All  compari¬ 
sons  were  non-significant.  Based  on  these  results,  the  two  cohorts  were 
combined  in  subsequent  analyses. 

Comparison  of  Pre-  and  Post-BSEP  Job 
Performance  RatingT 

The  average  Pre-  and  Post-BSEP  ratings  and  the  rating  gain  (Post-BSEP 
ratings  minus  Pre-BSEP  ratings)  are  presented  in  Table  3.  Although  38  state¬ 
ments  showed  some  gain,  only  15  were  statistically  Significant  using  Student's 
t-test.  The  four  irrelevant  statements  did  not  show  significant  gains,  indi¬ 
cating  that  the  supervisors  were  using  some  differentiation  in  their  ratings. 

The  five  statements  with  the  largest  gains  (starting  with  the  largest) 

were; 

•  Volunteers  to  help  others  who  are  having  difficulties 
in  tasks  requiring  mental  abilities  (rather  than 
mechanical  or  physical  abilities). 

a  Volunteers  for  job  assignments  that  require  mental 
abilities  (rather  than  mechanical  or  physical 
abilities). 

a  Uses  more  than  one  source  of  information  when  useful 
for  performing  job  assignments  or  tasks. 

a  Utilizes  maps  and/or  diagrams  correctly. 
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TABLE  3.  COMPARISON  OF  PRE-  AND  POST-BSEP 

JOB  PERFORMANCE  RATINGS  FOR  ALL  STATEMENTS 


Significance 


Statement 

N 

Pre-BSEP 

Mean* 

Post-BSEP 

Mean* 

Gain 

Level 

Of  Difference 

1 

65 

5.77 

5.94 

0.17 

N.S. 

2 

65 

5.75 

5.94 

0.18 

N.S. 

3 

65 

6.03 

6.20 

0.17 

N.S. 

4 

65 

5.49 

5.65 

0.15 

N.S. 

5 

65 

5.20 

5.54 

0.34 

<.05 

6 

65 

5.32 

5.58 

0.26 

<.05 

7 

65 

6.29 

6.43 

0.14 

N.S. 

8 

65 

5.77 

5.95 

0.18 

N.S. 

9 

65 

5.94 

6.05 

0.11 

N.S. 

10 

65 

5.35 

5.57 

0.22 

N.S. 

11 

64 

4.75 

5.22 

0.47 

<.01 

12 

65 

5.51 

5.92 

0.42 

<.01 

13 

65 

5.74 

5.98 

0.25 

<.05 

14 

65 

5.15 

5.49 

0.34 

<.05 

15 

64 

5.78 

5.86 

0.08 

N.S. 

16 

65 

6.23 

6.29 

N.S. 

17 

65 

5.98 

6.00 

mm 

N.S. 

18 

65 

5.80 

5.91 

0.11 

N.S. 

19 

65 

6.31 

6.22 

N.S. 

20 

64 

5.53 

5.84 

0.31 

N.S. 

21 

64 

5.05 

5.42 

0.38 

<.05 

22 

61 

6.03 

6.25 

0.21 

N.S. 

23 

65 

5.75 

6.02 

0.26 

<.05 

24 

65 

5.80 

6.05 

0.25 

N.S. 

25 

60 

5.37 

5.80 

0.43 

<.01 

26 

57 

6.26 

6.40 

27 

63 

5.44 

5.75 

28 

64 

5.73 

5.86 

0.12 

N.S. 

29 

62 

5.29 

5.79 

f'.50 

<.05 
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TABLE  3.  COMPARISON  OF  PRE-  AND  POST-BSEP 

JOB  PERFORMANCE  RATINGS  FOR  ALL  STATEMENTS 
(Continued) 


Statement 

N 

Pre-BSEP 

Mean* * 

Post-BSEP 

Mean* 

Gain 

Significance 

Level 

Of  Difference 

30 

64 

5.80 

6.02 

0.22 

N.S. 

31 

64 

5.48 

5.72 

0.23 

N.S. 

32 

64 

5.42 

5.88 

0.45 

<.01 

33 

63 

5.67 

5.94 

0.27 

<•05 

34 

64 

5.56 

5.78 

0.22 

N.S. 

35 

64 

5.62 

6.00 

0.38 

<.01 

36 

64 

5.77 

5.80 

0.03 

N.S. 

37 

64 

5.83 

6.00 

0.17 

N.S. 

38 

64 

5.78 

6.00 

0.22 

N.S. 

39 

64 

5.77 

6.05 

0.28 

<.05 

N.S.  -  not  significant 

*  -  based  on  7-point  scale 
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•  Completes  job  assignments  and  tasks  without  constant 
supervision. 

Without  a  control  group,  it  is  impossible  to  determine  if  the  reliable 
gains  obtained  are  due  to  repeated  measurement,  increased  job  experience, 
increased  maturity,  historical  artifacts,  BSEP,  or  BSEP  with  IE. 


Comparison  of  Pre-  and  Post-BSEP  Job  Performance 
Subscales  and  Total  Scores 


The  average  Pre-  and  Post-BSEP  subscales  and  total  scores,  and  the 
gains  (Post-BSEP  means  minus  Pre-BSEP  means)  are  presented  in  Table  4.  All 
subscales  and  the  total  score  showed  some  gain.  Most  gains  were  statistically 
significant  using  Student's  t-test.  For  the  total  score,  38  individual  ratings 
showed  a  positive  gain  (58%),  11  showed  no  gain  (17%),  and  16  showed  a  nega¬ 
tive  gain  (25%).  The  total  score  showed  a  significant  gain,  whereas  the 
irrelevant  items  subscale  did  not  show  a  significant  gain. 

The  five  subscales  with  the  largest  gains  (starting  with  the  largest) 

were: 

f  Confidence  with  Mental  Activities 

•  Planning 

•  Working  Independently 

•  Concentration 

t  Spatial  Orientation 

As  previously  noted,  it  is  difficult  to  interpret  these  gains  without  a  control 
gt-oup.  However,  these  subscales  represent  areas  to  which  IE  is  addressed. 

Analysis  of  the  Open-Ended  Question 

Responses  to  the  open-ended  question  at  the  end  of  the  Post-BSEP 
questionnaire  indicated  both  the  presence  or  absence  of  performance  differ¬ 
ences,  and  the  area  of  improvement.  Table  5  presents  the  supervisor's  judg¬ 
ments  of  performance  differences.  A  majority  of  supervisors  that  responded 
indicated  "general  improvement".  Table  6  presents  the  areas  of  improvement. 

Job  performance  and  Confidence  were  most  often  mentioned.  This  agrees  with 
the  more  detailed  questionnaire  results. 
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TABLE  4.  COMPARISON  OF  PRE-  AND  POST-BSEP  JOB  PERFORMANCE 
RATINGS  FOR  SUBSCALES  AND  TOTAL  SCORE 


Subscale 

N 

Pre-BSEP 

Mean 

Post-BSEP 

Mean 

Gain 

Significance 

Level 

Of  Difference 

Problem  Solving 

64 

5.61 

5.82 

0.21 

<.05 

Concentration 

64 

5.57 

5.83 

0.26 

<.05 

Verbal  Expression 
(Categorization/ 

Naming) 

64 

5.62 

5.84 

0.22 

<.05 

Learning  New  Material 

65 

5.75 

5.88 

0.13 

N.S. 

Planning  (Time  Orienta¬ 
tion) 

63 

5.26 

5.59 

0.33 

<.01 

Spatial  Orientation 

57 

5.89 

6.14 

0.25 

<.01 

Following  Instructions 

65 

5.76 

5.98 

0.22 

N.S. 

Understanding  Relation¬ 
ships  (Comparisons) 

64 

5.78 

6.02 

0.24 

<.01 

Checking  Own  Work 

64 

5.53 

5.70 

0.17 

N.S. 

Confidence  with  Mental 
Activities  (Volunteering 
for  Mental  Tasks) 

61 

5.03 

5.52 

0.49 

<.01 

Working  Accurately 

63 

5.79 

5.81 

0.02 

N.S. 

Using  Information 
(Illustrations) 

64 

5.70 

5.93 

0.23 

<.05 

Working  Independently 

63 

5.68 

5.94 

0.26 

<.05 

Total  Score 

65 

5.62 

5.86 

0.24 

<.01 

Irrelevant  Items 

57 

6.05 

6.18 

0.13 

N.S. 
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TABLE  5.  SUPERVISORS'  JUDGMENT  OF  PERFORMANCE  DIFFERENCES 


Judged  Change  1n 
Soldier's  Performance 

Number  of 
Supervisors 

Percent  of 
Supervi sors 
(N  =  65) 

Percent  of 
Supervi sors 
Responding 
(N  »  54) 

No  Difference 

15 

23.1 

27.8 

Degraded  Performance 
or  Attitude 

4 

6.2 

7.4 

Improvement  Due  to 

Job  Experience 

1 

1.5 

1.9 

General  Improvement. 

28 

43.1 

51.8 

No  Indication  of 

Change 

6 

9.2 

11.1 

No  Response 

n 

16.9 

TABLE  6.  SUPERVISORS'  JUDGMENTS  OF  AREAS  OF  IMPROVEMENT 


Area  of  Improvement 

Number  of 
Responses* 

Percent  Of 
Responses 

Job  Performance/Productivity 

7 

17.9 

Confidence/Helping  Others 

7 

17.9 

Understanding  of  Job/ Equipment 

5 

12.8 

Attitude/Interest 

4 

10.3 

Mental  Performance 

4 

10.3 

Communication  (Wri ti ng/Speaking) 

3 

7.7 

Reading 

3 

7.7 

Planning/Use  of  Time 

2 

5.1 

Accuracy 

2 

5.1 

Responsibility 

2 

5.1 

Total 

39 

100.0 

*  26  supervisors  responded,  13  gave  two  responses 


f 

! 

I 


23 


Comparison  of  Supervisors'  Judgments 
with  Job  Performance  Rat'tn'g? 

The  supervisors  were  divided  into  two  groups;  those  who  judged  that 
the  soldier's  job  performance  hadn't  changed  or  was  degraded  compared  to  Pre- 
BSEP  performance  (N=19)  and  those  who  judged  that  the  soldier's  job  performance 
had  shown  general  improvement  (N=28). 

The  gains  in  job  performance  subscales  and  total  score  (Post-BSEP 
means  minus  Pre-BSEP  means)  were  compared  for  these  two  groups  of  supervisors. 
The  purpose  of  the  analysis  was  to  determine  if  the  ratings  reflected  the 
supervisors'  overall  judgments  of  performance  differences.  Generally,  job 
performance  ratings  were  lower  after  BSEP/IE  if  the  supervisors  judged  that 
performance  was  unchanged  or  degraded  (i.e. ,  mean  gains  were  usually  nega¬ 
tive),  and  ratings  were  higher  after  BSEP/IE  if  the  supervisors  judged  that 
performance  was  improved  (i.e.,  mean  gains  were  positive).  The  statistical 
significance  of  the  differences  between  the  two  groups  of  supervisors  was 
evaluated  using  Student's  t-test.  Gains  were  significantly  different  between 
the  two  groups  of  supervisors  on  11  of  the  14  subscales  (including  the 
irrelevant  items  subscale)  and  on  total  score.  These  findings  indicate  that 
the  supervisors'  job  performance  ratings  agreed  with  their  subsequent  judg¬ 
ments  of  overall  job  performance  change. 
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CONCLUSIONS  AND  RECOWENDATIONS 

Based  on  a  comparison  of  Fall,  1980  and  Fall,  1981  Fort  Knox  8SEP 
participants,  it  appears  that  IE  resulted  in  a  small  but  reliable  difference 
in  Total  Battery  TABE  scores.  This  difference  was  largely  due  to  improved  per¬ 
formance  in  the  area  of  Language  Mechanics  and  Expression,  compared  to  the 
control  group  without  IE.  The  lack  of  an  experimental  and  control  group  formed 
through  random  assignment  makes  this  conclusion  tentative. 

Only  about  two  hours  per  day  for  six  weeks  was  devoted  to  IE  by  the 
BSEP/IE  participants  in  this  study.  Each  participant  did  not  spend  that  same 
amount  of  time  on  IE  activities  nor  complete  all  IE  books.  Instrumental  enrich¬ 
ment  is  intended  to  be  of  much  longer  duration.  In  future  evaluations,  con¬ 
sideration  should  be  given  to  increasing  the  amount  and  duration  of  IE  activities, 
and  monitoring  and  recording  these  activities  more  systematically. 

No  firm  conclusions  about  the  effects  of  IE  on  job  performance  can 
be  made,  due  to  the  absence  of  a  control  group,  and  the  subjective  and  retro¬ 
spective  nature  of  the  job  performance  measures  themselves.  The  job  performance 
measures  (supervisor  ratings)  suggest  that  improvements  occured  in  confidence 
with  mental  activities,  planning,  working  Independently,  concentration,  and 
spatial  orientation.  These  are  areas  to  which  IE  is  addressed. 

Firm  conclusions  about  the  learning  strategies  training  program  will 
not  be  possible  without  a  more  carefully  planned  and  controlled  evaluation 
study.  The  present  study  was  preliminary  in  nature.  Since  it  appears  that  IE 
had  some  impact  on  BSEP  performance,  it  is  important  to  conduct  a  more  rigor¬ 
ously  controlled  study.  A  conceptual  plan  for  such  a  study  is  described  in 
the  next  section. 


SUGGESTED  PLAN  FOR  EVALUATING  FUTURE  IE  PROGRAMS 

The  evaluation  of  IE  presented  in  this  report  was  hampered  by  the 
lack  of  an  adequate  control  group  and  a  short  project  time  schedule.  To  over¬ 
come  these  problems  a  plan  was  developed  to  evaluate  future  IE  programs.  This 
plan  is  presented  below. 

Purpose 

The  purpose  of  the  study  would  be  to  determine  the  effect  of  IE  on 
BSEP  performance  and  subsequent  job  performance.  If  IE,  in  fact,  improves 
cognitive  skills,  then  BSEP  performance  as  measured  by  TABE,  should  be 
improved.  Further,  if  cognitive  skills  are  improved,  then  changes  in  certain 
aspects  of  job  performance  should  also  be  improved.  This  study  would  determine 
if  IE  improves  soldier  BSEP  scores  and  job  performance. 

Study  Design 

The  basic  design  of  the  study  would  be  a  comparison  of  soldiers  who 
complete  BSEP  with  IE  with  soldiers  who  complete  BSEP  without  an  IE  component. 
Two,  six-  or  eight-week  BSEP  programs  would  be  run  concurrently.  These  two 
programs  would  be  taught  under  essentially  identical  conditions  except  one 
program  would  include  IE  material.  How  the  IE  is  to  be  incorporated  into  BSEP 
and  scheduled  should  be  determined  through  consultation  with  IE  specialists. 
Ideally,  the  two  BSEP  courses  would  contain  the  same  amount  of  BSEP  material 
with  one  having  the  additional  IE  material.  However,  this  will  mean  that  one 
course  may  be  four  hours  per  day  and  the  other  five  to  six  hours  per  day.  Other 
alternatives  are  possible  and  how  these  courses  are  to  be  composed  and  adminis¬ 
tered  will  have  to  be  determined  through  discussions  with  BSEP  administrators 
and  IE  specialists.  It  is  suggested,  however,  that  consideration  be  given  to 
presenting  IE  over  a  longer  duration  than  in  the  recent  Fort  Knox  programs. 

As  created  by  Feuerstein,  the  IE  program  consists  of  15  instruments  that  pro¬ 
vide  material  for  three  to  five  one-hour  lessons  per  week  for  two  to  three 
years  (Hobbs,  1980). 

Soldiers  eligible  to  take  BSEP  would  be  randomly  assigned  in  equal 
numbers  to  the  two  courses.  These  groups  would  also  be  separated  into  classes 
to  adjust  to  typical  teacher-student  loads, 
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Prior  to  beginning  the  BSEP  program,  supervisors  of  all  the  soldiers 
in  both  groups  would  be  identified  and  surveyed  using  a  questionnaire  much 
like  the  one  used  in  the  study  reported  herein.  The  questionnaire  would  instruct 
the  supervisors  to  rate  the  soldier  on  a  series  of  behavioral  statements 
related  to  job  performance.  Approximately  eight  weeks  after  completing  BSEP, 
the  same  questionnaire  would  be  sent  to  the  same  supervisors  to  obtain  a  judg¬ 
ment  of  the  soldiers'  Post-BSEP  job  performance.  The  questionnaire  would 
include  some  open-ended  questions  relevant  to  change  versus  no  change  in  job 
performance  as  a  result  of  BSEP  or  BSEP  plus  IE.  Finally,  objective  measures 
of  job  performance  such  as  the  Enlisted  Evaluation  Report  (EER)  or  SQT  (Skill 
Qualification  Test)  would  also  be  collected  for  both  groups. 

Pre-  and  Post-BSEP  TABE  scores  (as  well  as  other  data)  would  be 
collected  for  soldiers  in  both  groups  and  comparisons  made.  The  questionnaire 
data  and  objective  measures  of  job  performance  would  also  be  compared  for  the 
two  groups  and  all  sets  of  data  would  be  compared  to  each  other  with  respect  to 
logical  interrelationships. 

The  advantages  of  this  study  design  are  that  all  soldiers  would 
receive  essentially  the  same  amount  of  BSEP,  the  control  group  would  provide  a 
technically  sound  basis  for  comparison,  and  the  supervisors  could  make  judg¬ 
ments  about  the  soldiers  at  the  appropriate  points  in  time  (i.e. ,  Pre-  and 
Post-BSEP). 

Data  Collection 

The  data  to  be  collected  would  be  Pre-  and  Post-BSEP  TABE  scores  and 
Pre-BSEP  GT  scores  for  all  soldiers,  as  well  as  basic  demographic  and  service 
data.  In  addition,  for  each  soldier  receiving  IE,  a  record  would  be  kept  of 
the  amount  of  class  time  devoted  to  IE  and  the  IE  instruments  completed  or 
partially  completed.  These  data  are  needed  to  assess  the  implementation  of  IE 
and  would  be  obtained  from  the  BSEP  administrators. 

For  data  collection  concerning  job  performance,  the  Pre-  and  Post- 
BSEP  questionnaires  would  be  distributed  to  the  soldiers'  supervisors.  The 
Pre-BSEP  questionnaire  would  be  distributed  just  prior  to  the  soldiers'  parti¬ 
cipation  in  the  program.  The  Post-BSEP  questionnaire  would  be  distributed 
approximately  eight  weeks  after  the  soldier  finishes  BSEP.  Objective  measures 
of  job  performance  would  be  obtained  from  MILPERCEN  (Military  Personnel  Center) 
or  EREC  (Enlisted  Records  Evaluation  Center). 
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The  procedure  for  distributing  the  questionnaires  should  be  closely 
coordinated  with  the  involved  Unit  Conmanders  in  order  to  achieve  the  best 
return  rate. 

Data  Analysis  Plans 

The  data  analysis  would  consist  of  a  statistical  comparison  of  Post- 
BSEP  TABE  scores  for  the  BSEP  group  and  the  BSEP/IE  group,  while  controlling 
6T  and  Pre-TABE  scores.  Analysis  of  covariance  or  regression  models  would  be 
used  to  analyze  the  data. 

The  questionnaire  analysis  would  consist  of  comparing  the  results  of 
the  Pre-  and  Post-BSEP  questionnaires  to  determine  whether  statistically  signi¬ 
ficant  improvement  in  job  performance  had  occurred  between  the  two  groups.  In 
addition,  areas  of  improved  cognitive  skill  and  job  performance  would  be 
identified.  The  two  groups  would  also  be^compared  through  a  statistical 
analysis  of  the  objective  measures  of  performance. 

Finally,  cross  comparisons  between  the  questionnaire  data  and  the 
TABE  data  would  be  made  to  determine  whether  TABE  subscale  scores  are  related 
or  predictive  of  areas  of  improved  job  performance,  and  to  identify  other 
interrelationships. 


REFERENCES 


Hobbs,  Nicholas.  Feuerstein's  Instrumental  Enrichment; 
Teaching  Intelligence  to  Adolescents.  Educational  Leader¬ 
ship,  April  1980. 


Anon.  Instrumental  Enrichment:  A  Curriculum  Intervention 
for  Adolescents  who  are  Deficient  In  Thinking  and  Learning 
Skills.  Curriculum  Development  Associates,  unpublished, 
non-dated  paper. 


Link,  Frances  R.  Instrumental  Enrichment  Demonstration 
Program  conducted  at  Fort  Knox,  Kentucky,  Needs  Assessment 
Report.  Contract  No.  DABT58-81-C-0085,  July  1981. 


29 


APPENDIX  I 


JOB  STATEMENTS  FOR  COGNITIVE/PERFORMANCE  PARAMETERS* 


Problem  Solving 


•  Performs  job  assignments  and  tasks  carefully  and 
completely. 

•  Performs  job  assignments  and  tasks  In  a  systematic, 
logical  and  efficient  sequence. 

•  Performs  job  assignments  and  tasks  correctly. 

•  Solves  unexpected  problems  that  occur  while  perform¬ 
ing  job  assignments  or  tasks. 

•  Uses  more  than  one  source  of  Information  when  useful 
for  performing  job  assignments  or  tasks. 

•  Selects  and  uses  the  correct  Information,  tools, 
and  equipment  that  is  needed  to  perform  the  job 
assignments  or  tasks. 

•  Is  flexible  in  handling  new  situations  and  job 
assignments. 


Accuracy  and  Precision 


•  Tries  to  be  precise  when  needed  to  carry  out  job 
assignments  and  tasks. 

•  Tries  to  be  accurate  In  spelling,  arithmetic,  and 
following  instructions. 


Verbal  Naming 


•  Expresses  thoughts  adequately  about  job  activities 
and  problems. 

•  Uses  correct  names  for  Army  equipment,  tasks,  and 
places. 


*  These  statements  were  used  as  the  basis  for  developing  the  Pre-  and 
Post-BSEP  questionnaires  completed  by  the  supervisors. 
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Learning  New  Tasks 


•  Learns  material  from  "ms,  SMs,  and  FMs  and  other 
written/ visual  sources  like  TEC. 

•  Learns  from  job  experiences. 

Planning 


•  Thinks  through  and  prepares  for  job  assignments 
carefu'^ly  and  completely. 

•  Thinks  through  and  prepares  for  job  assignments 
systematically  and  logically. 

•  Prepares  for  job  assignments  well  ahead  of  schedule. 


Spatial  Orientation  and  Direction 


a  Gets  to  parts  of  the  Post  without  asking  directions. 

•  Identifies  North,  South,  East  and  West  correctly. 

•  Understands  how  parts  of  equipment  used  on  the  job 
fit  or  connect  together. 

•  Utilizes  maps  and/or  diagrams  correctly. 

Following  Instructions 

t  Follows  supervisor's  instructions  the  first  time 
in  carrying  out  job  assignments  and  tasks. 

t  Follows  procedures  given  in  TMs,  SMs,  and  FMs. 


Relationships 


•  Understands  how  parts  of  equipment  used  on  the 
job  work  together. 

•  Understands  why  job  assignments  and  tasks  are 
performed  to  achieve  an  objective  or  goal. 

•  Understands  relationships  and  responsibilities  of 
the  different  duty  positions  of  the  unit. 
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Checks  Own  Work 


•  Checks  own  work  upon  completion  of  a  job  or  task 
using  an  appropriate  method. 

•  Makes  sure  that  the  quality  of  his  or  her  work  Is 
adequate. 


Volunteers 


•  Volunteers  for  job  assignments  that  require  mental 
abilities  (rather  than  mechanical  or  physical 
abilities). 

•  Volunteers  to  help  others  who  are  having  difficul¬ 
ties  In  tasks  requiring  mental  abilities  (rather 
than  mechanical  or  physical  abilities). 


Attention  and  Concentration 


•  Completes  job  assignments  and  tasks  without 
constant  supervision. 

•  Completes  job  assignments  without  daydreaming, 
excessive  work  breaks,  talking  with  others,  etc. 

e  Concentrates  while  working  on  job  assignments  or 
tasks. 

•  Pays  attention  to  detail  in  performing  job 
assignments  or  tasks. 


Independence 


e  Performs  Individual  job  assignments  or  tasks  with 
minimum  help. 

•  Tries  to  perform  job  assignments  or  tasks  before 
asking  for  help. 


32 


Irrelevant 


•  Maintains  a  neat  and  well  groomed  appearance. 

•  Gets  along  well  with  soldiers  in  his  or  her  unit. 

•  Keeps  quarters  clean  and  orderly, 
e  Shows  high  morale. 
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APPENDIX  II.  QUESTIONNAIRES 


UNIT  COMMANDER/OFFICE  INSTRUCTIONS 


Battelle  Laboratories  is  under  contract  with  the  Army  to  study 
the  effects  of  educational  programs  on  soldier  job  performance.  To  do 
this  we  are  surveying  supervisors  of  a  sample  of  soldiers.  Each  of  the 
enclosed  questionnaires  is  identified  with  a  soldier's  name.  Please 
distribute  the  attached  questionnaires  to  those  in  your  unit  who  were 
the  immediate  supervisors  of  these  soldiers  from  October,  1981  through 
February,  1982.  It  is  important  that  the  questionnaires  be  distributed 
only  to  supervisors  who  had  an  opportunity  to  observe  the  soldier's  per¬ 
formance  from  October,  1981  through  February,  1982.  In  other  words,  each 
soldier's  questionnaire  should  be  completed  by  his  or  her  immediate 
supervisor  who  was  in  charge  of  the  soldier  during  this  period. 

Please  d’Stribute  the  questionnaires  early  this  morning.  The 
questionnaires  should  be  returned  to  you  and  you  should  send  them  to 
Jim  Jones  at  the  Education  Center.  A  second  and  final  questionnaire 
(similar  in  content  and  length)  will  also  be  distributed  tomorrow  to  the 
same  group  of  supervisors.  Please  keep  all  completed  questionnaires  in 
confidence. 
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SOLDIER  SUPERVISOR'S  QUESTIONNAIRE:  I 


Soldier: 

cc 

1-3 

Unit: 

cc 

4-6 

Supervisor  (please  print): 

cc 

7-9 

INSTRUCTIONS 


As  the  supervisor  of  the  soldier  identified  above  you  are  being 
asked  to  make  judgements  about  this  soldier  during  October,  1981.  When 
making  your  judgements,  be  sure  to  think  back  to  the  soldier's  performance 
during  October,  1981.  First,  print  your  name  above.  Then  please  make 
judgements  about  the  soldier's  performance  on  a  scale  of  1  to  7.  On  the 
next  page  read  the  statement  in  the  left  column  and  then  circle  one  of  the 
seven  numbers  depending  on  your  judgement.  Please  judge  the  soldier  on 
every  statement  to  the  best  of  your  knowledge  of  the  soldier's  performance. 
You  will  be  given  the  remainder  of  the  day  to  complete  this  questionnaire. 
Please  return  the  questionnaire  to  the  Unit  Office  by  3:00  p.m.  Tomorrow 
you  will  receive  one  more  questionnaire  of  similar  content  and  length. 

Your  completion  of  this  questionnaire  is  voluntary.  Your 
answers  will  be  kept  in  complete  confidence  and  will  not  become  a  part  of 
any  permanent  record.  The  information  will  be  used  for  research  purposes 
only.  This  is  not  a  test  of  you  or  the  soldier  you  are  rating. 


Rating  of  Performance 

in 

October, 

1981 
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1.  Performs  job  assignments  and  tasks 
correctly. 

2.  Concentrates  while  working  on  job 
assignments  or  tasks. 

3.  Maintains  a  neat  and  well 
groomed  appearance. 

4.  Expresses  thoughts  adequately 
about  job  activities  and  problems. 

5.  Learns  material  from  TMs,  SMs,  and 
FMs  and  other  written/visual 
sources  like  TEC. 

6.  Thinks  through  and  prepares  for 
job  assignments  carefully  and 
completely. 

7.  Gets  to  parts  of  the  Post  without 
asking  for  directions. 

8.  Follows  supervisor's  instructions 
the  first  time  in  carrying  out 
job  assignments  and  tasks. 

9.  Understands  how  parts  of  equip¬ 
ment  used  on  the  job  work 
together. 

10.  Checks  own  work  upon  completion  of 
a  job  or  task  using  an  appropriate 
method. 


1  2  3  4  5  6  7 

1  2  3  4  5  6  7 

1  2  3  4  5  6  7 

1  2  3  4  5  6  7 

1  2  3  4  5  6  7 

1  2  3  4  5  6  7 

1  2  3  4  5  6  7 

1  2  3  4  5  6  7 
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cc  IS 


Rating  of  Performance  in 
_ October.  1981 _ 
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11.  Volunteers  for  job  assignments 
that  require  mental  abilities 
(rather  than  mechanical  or 
physical  abilities). 

12.  Completes  job  assignments  and 
tasks  without  constant  supervi¬ 
sion. 

13.  Performs  individual  job  assign¬ 
ments  or  tasks  with  minimum 
help. 

14.  Solves  unexpected  problems  that 
occur  while  performing  job  assign¬ 
ments  or  tasks. 

15.  Tries  to  be  accurate  in  spelling, 
arithmetic,  and  following  instruc¬ 
tions. 

16.  Gets  along  with  soldiers  in  his 
or  her  unit. 

17.  Selects  and  uses  the  correct 
information,  tools,  and  equip¬ 
ment  that  are  needed  to  perform 
job  assignments  and  tasks. 

18.  Performs  job  assignments  and  tasks 
carefully  and  completely. 

19.  Learns  from  job  experiences. 


1  2  3  4  5  6  7 
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1  2  3  4  5  6  7 


1  2  3  4  5  6  7 

1  2  3  4  5  6  7 


1  2  3  4  5  6  7 

1  2  3  4  5  6  7 


1  2 


3  4 


7 


1  2  3  4  5  6  7 

1  2  3  4  5  6  7 


20.  Is  flexible  in  handling  new  situa 
tions  and  job  assignments. 


1  2  3  4  5  6  7 
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Almost  Always 
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21.  Prepares  for  job  assignments  well 
ahead  of  schedule. 


2  3  4  5  6  7 


22.  Identifies  directions  such  as  North, 


South,  East  and  West  correctly.  1 

23.  Follows  procedures  given  in  TMs, 

SMs,  and  FMs.  1 

24.  Understands  the  relationships  and 
responsibilities  of  the  different 

duty  positions  in  the  unit.  1 

25.  Utili2es  maps  and/or  diagrams 

correctly.  1 

26.  Keeps  quarters  clean  and  orderly.  1 

27.  Thinks  through  and  prepares  for 
job  assignments  systemrtically 

and  logically.  1 

28.  Makes  sure  that  the  quality  of 

his  or  her  work  is  adequate.  1 


2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 


4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 


6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 


29.  Volunteers  to  help  others  who  are 
having  difficulties  in  tasks 
requiring  mental  abilities  (rather 
than  mechanical  or  physical 

abilities).  1234567 

30.  Understands  how  parts-  of. 
equipment  used  on  the  job  fit  or 

connect  together.  1  234567 


38 


Rating  of  Performance  in 
_ October.  1981 _ 
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31.  Completes  job  assignments  without 
daydreaming,  excessive  work  breaks, 
talking  with  others,  etc. 

1  2 

3 

4 

5 

6 

7 

32.  Uses  more  than  one  source  of  infor¬ 
mation  when  useful  fo>  performing 
job  assignments  or  tasks. 

1  2 

3 

4 

5 

6 

7 

33.  Tries  to  perform  job  assignments 

or  tasks  before  asking  for  help. 

1  2 

3 

4 

5 

6 

7 

34.  Pays  attention  to  detail  in  per- 

forming  job  assignments  or  tasks. 

1  2 

3 

4 

5 

6 

7 

35.  Understands  why  job  assignments 
and  tasks  are  performed  to  achieve 
an  objective  or  goal. 

1  2 

3 

4 

5 

6 

7 

36.  Tries  to  be  precise  when  needed  to 
carry  out  job  assignments  and 
tasks. 

1  2 

3 

4 

5 

6 

7 

37.  Shows  high  morale. 

1  2 

3 

4 

5 

6 

7 

38.  Performs  job  assignments  and  tasks 

carefully  and  completely. 

1  2 

3 

4 

5 

6 

7 

39.  Uses  correct  names  for  Amw  equip- 

ment,  tasks  and  places. 

1  2 

3 

4 

5 

6 

7 

Thank  you  for  your  cooperation.  Please  go  back  and  make  certain  that  you 

have 

answered  every  question. 

Almost  Always 


UNIT  COMMANDER/OFFICE  INSTRUCTIONS  -  DAY  II 


Please  distribute  this  second  and  last  group  of  ques¬ 
tionnaires  to  the  same  supervisors  as  the  first  group  of 
questionnaires. 

Please  distribute  the  questionnaires  this  morning, 

1f  possible.  The  questionnaires  should  be  returned  to  you  and 
you  should  send  them  to  Jim  Jones  at  the  Education  Center. 
Please  keep  all  completed  questionnaires  In  confidence. 

It  Is  Important  that  you  collect  all  questionnaires 
as  soon  as  possible  and  send  them  to  Jim  Jones  at  the  Educa¬ 
tion  Center. 


THANK  YOU  FOR  YOUR  HELP  AND  COOPERATION 


SOLDIER  SUPERVISOR'S  QUESTIONNAIRE:  II 


Soldier: 

Unit- 


Supervisor  (please  print) 


INSTRUCTIONS 


As  the  supervisor  of  the  soldier  identified  above,  you  are  being 
asked  to  make  judgements,  and  answer  a  question,  about  this  soldier's 
current  performance.  Fill  out  this  questionnaire  just  as  you  did  the  pre- 
vious  one,  plus  answer  the  question  at  the  end.  Be  sure  to  judge  the 
soldier's  behavior  on  his  or  her  current  performance.  You  will  be  given 
the  remainder  of  the  day  to  complete  the  questionnaire.  Please  return 
this  questionnaire  to  the  Unit  Office  by  3:00  p.m. 

As  in  the  previous  questionnaire,  your  completion  of  the  question¬ 
naire  is  voluntary.  Your  answers  will  be  kept  in  complete  confidence  and 
will  not  become  a  part  of  any  permanent  record.  The  information  will  be 
used  for  research  purposes  only.  This  is  not  a  test  of  you  or  the  soldier 
you  are  rating. 

It  is  very  important  that  you  complete  this  questionnaire  as 
soon  as  possible  ana  send  it  back  to  the  Unit  Office  inimediatelj^ 
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Rating  of  Current  Perfonnance 
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1. 

Performs  job  assignments  and  tasks 
correctly. 

1 

2 

3 

4 

5 

6 

7 

cc  50 

2. 

Concentrates  while  working  on  job 
assignments  or  tasks. 

1 

2 

3 

4 

5 

6 

7 

3. 

Maintains  a  neat  and  well 
groomed  appearance. 

1 

2 

3 

4 

5 

6 

7 

4. 

Expresses  thoughts  adequately 
about  job  activities  and  problems. 

1 

2 

3 

4 

5 

6 

7 

5. 

Learns  material  from  TMs,  SMs,  and 

FMs  and  other  written/ visual 
sources  like  TEC. 

1 

2 

3 

4 

5 

6 

7 

6. 

Thinks  through  and  prepares  for 
job  assignments  carefully  and 
completely. 

1 

2 

3 

4 

5 

6 

7 

7. 

Gets  to  parts  of  the  Post  without 
asking  for  directions. 

1 

2 

3 

4 

5 

6 

7 

8. 

Follows  supervisor's  instructions 
the  first  time  in  carrying  out 
job  assignments  and  tasks. 

1 

2 

3 

4 

5 

6 

7 

9. 

Understands  how  parts  of  equip¬ 
ment  used  on  the  job  work 
together. 

1 

2 

3 

4 

5 

6 

7 

10. 

Checks  own  work  upon  Completion  of 
a  job  or  task  using  an  appropriate 
method. 

1 

2 

3 

4 

5 

6 

7 

cc  55 

L 
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Almost  Always 
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n. 

Volunteers  for  job  assignments 
that  require  mental  abilities 
(rather  than  mechanical  or 
physical  abilities). 

1 

2 

3 

4 

5 

6 

7 

12. 

Completes  job  assignments  and 
tasks  without  constant  supervi* 

Sion. 

1 

2 

3 

4 

5 

6 

7 

13. 

Performs  individual  job  assign¬ 
ments  or  tasks  with  minimum 
help. 

1 

2 

3 

4 

5 

6 

7 

14. 

Solves  unexpected  problems  that 
occur  while  performing  job  assign¬ 
ments  or  tasks. 

1 

2 

3 

4 

5 

6 

7 

15. 

Tries  to  be  accurate  in  spelling, 
arithmetic,  and  following  instruc¬ 
tions. 

1 

2 

3 

4 

5 

6 

7 

16. 

Gets  along  with  soldiers  in  his 
or  her  unit. 

1 

2 

3 

4 

5 

6 

7 

17. 

Selects  and  uses  the  correct 
information,  tools,  and  equip¬ 
ment  that  are  needed  to  perform 
job  assignments  and  tasks. 

1 

2 

3 

4 

5 

6 

7 

18. 

Performs  job  assignments  and  tasks 
carefully  and  completely. 

1 

2 

3 

4 

5 

6 

7 

19. 

Learns  from  job  experiences. 

1 

2 

3 

4 

5 

6 

7 

20. 

Is  flexible  in  handling  new  situa¬ 
tions  and  job  assignments. 

1 

2 

3 

4 

5 

6 

7 
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21.  Prepares  for  job  assignments  well 
ahead  of  schedule. 

22.  Identifies  directions  such  as  North, 
South,  East  and  West  correctly. 


Rating  of  Current  Performance 
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1  2  3  4  5  6  7 

1  2  3  4  5  6  7 


23.  Follows  procedures  given  in  TMs, 
SMs,  and  FMs. 


2  3  4  5  6  7 


24.  Understands  the  relationships  and 
responsibilities  of  the  different 

duty  positions  in  the  unit.  1234567 


25.  Utilizes  maps  and/or  diagrams 
correctly. 


2  3  4  5  6  7 


26.  Keeps  quarters  clean  and  orderly. 

27.  Thinks  through  and  prepares  for 
job  assignments  systematically 
and  logically. 

28.  Makes  sure  that  the  quality  of 
his  or  her  work  is  adequate. 

29.  Volunteers  to  help  others  who  are 
having  difficulties  in  tasks 
requiring  mental  abilities  (rather 
than  mechanical  or  physical 
abilities). 

30.  Understands  how  parts-  of 
equipment  used  on  the  job  fit  or 
connect  together. 
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Almost  Always 


31.  Completes  job  assignments  without 
daydreaming,  excessive  work  breaks, 
talking  with  others,  etc. 

32.  Uses  more  than  one  source  of  Infor¬ 
mation  when  useful  for  performing 
job  assignments  or  tasks. 

33.  Tries  to  perform  job  assignments 
or  tasks  before  asking  for  help. 

34.  Pays  attention  to  detail  In  per¬ 
forming  job  assignments  or  tasks. 

35.  Understands  why  job  assignments 
and  tasks  are  performed  to  achieve 
an  objective  or  goal. 

36.  Tries  to  be  precise  when  needed  to 
carry  out  job  assignments  and 
tasks. 

37.  Shows  high  morale. 

38.  Performs  job  assignments  and  tasks 
carefully  and  completely. 

39.  Uses  correct  names  for  Army  equip¬ 
ment,  tasks  and  places. 


Rating  of  Current  Performance 
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Question: 

How  would  you  describe  the  difference  in  this  soldier's  performance 
between  October  of  1981  and  the  past  few  weeks?  For  example,  what  differences 
have  you  noticed  in  mental  performance,  and  in  job  performance? 


Thank  you  for  your  cooperation.  Please  go  back  and  make  certain  that  you  have 
answered  every  question. 


END 

DATE 


